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ABSTRACT 



Technology education planners must evaluate, change, and 
modify content to match rapid international advancements. Verification of 
seen and unseen factors that would improve technology education output is 
possible through periodic assessment of student learnability competence. The 
ultimate tool in researching student learnability is the assessment 
competency of the teacher. A topical outline has been suggested for a course 
resulting in competence in learnability assessment. Possible objectives have 
been developed. The content of such a course would include the required three 
desirable attributes of educational and psychological measuring devices: 
validity, reliability, and usability. Observations by teachers should get at 
typical student behaviors in the affective domain. Assessment competency can 
be completed either by formal testing with published and teacher-made tests 
or by observing and scaling students’ typical classroom behavior. 
Recommendations for technology education planning through the intermediate 
step of learnability assessment competence are: planning to meet objectives; 
students' computerized academic behavior and fruitfulness of assessment; 
assessment of immediate practical needs; straightening technology education 
curriculum; required ongoing assessment; direct access to technology 
education resources; and technology advancement and nonstop planning. 
(Contains 23 references.) ( YLB) 



Reproductions supplied by EDRS are the best that can be made 
from the original document. 



Abstract 



Leamability Vs Assessment Competency as a Constitutive step for 
Planning improvement in Technology Education 
Dr Saeed T. Al-Mallah & Dr. Abdulaziz Al-Malik 

Assessment Competency has been an ultimate tool in researching student 
academic attitude in Higher Education in general and very recently becomes as an 
effective technical requirement in planning improvement in Technology Education. 
Anne E. Edwards and Peter Knight London (1995); Brown S. and N. Peter (1994). 
This overcomes the essential need for planning through the constructivism in 
Technology Education. ERIC Ref. 378.(1999-2000). Periodic assessment for instance 
in Colleges of Technology in Saudi Arabia based on student leamability reflects the 
above-mentioned competences required by higher education, industry and business like. 
This paper tempts to verity the seen and unseen factors of Technology Education output to 
facilitate assessment, remedy and modification processes to cope with the advancement of 
technology. Rudner, L.M. and Boston, C. (1994,1998). This would help us at the end 
investigate the link between planning improvement and situational circumstantial 
methods of increasing Leamability. (Winter, 1993b) to give a better output in 
Technology Education Planning. (TEP). 
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Leamability vs assessment competency as a constitutive step for planning 
improvement in Technology Education 
Dr. Saeed T. Al-Mallah & Dr. Abdulaziz Al-Malik 

Attention to competences in leamability and assessment linked to the need of the immediate 
users of the products of higher education advantages graduates who enter the job market. So 
training students to be able to analyze their rich experiences in order to identify competences 
relevant to specific fields may be ultimately more conductive to creating the autonomous, 
self-motivated learner required by higher education, industry and business like. Anne E. 
Edwards and Peter Knight London (1995); Brown S. and N. Peter (1994). 

This research attempts to investigate the powerfulness of planning throughout the 
graph of student leamability in various times to understand the seen and unseen 
factors for Technology Education output in order to make it easy for planners to 
evaluate, change and modify the contents in a way that matches the rapid international 
advancement Rudner, L.M. and Boston, C. (1994;1998). Newman, F. M. Marks, H. 
M. and Gamoran, R. “Standards that Boost student Performance”, (1995), does help 
us in this regard to do more refinement, which directs the research to a new dimension 
to understand the relation between planning development and situational 
circumstantial methods of increasing leamability. (Winter, 1993b). 

The whole idea of the research is projected in the following chart. 









Planning 
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Tech. Education 












Situational circumstantial methods 
of increasing leamability 








Constructive stations of competency 


in leamability 




Seen vs unseen factors of improvement Technical Education 



The four steps are to be carried out in a systematic sequence as per the following 
explanations: 

1 . Planning improvement in Tech. Education... using all the applicable devices that improve planning. 

2. Situational circumstantial methods of increasing leamability. . .controlling all possible situations. 

3. Constructive stations of competency in Leamability... including all positive forwarding steps in 
learning. 

4. Seen vs unseen factors of improvement Technical Education ... deciding the harmony between 

them. 
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The Research Claim: 



• Why are we sometimes faced by less fruitful results out of our planning in 
Technology Education although, we always put more effort? 

If so, what is the shortest way to maintain the appropriate remedies? 

• Teaching staff is the key factor to planning improvement in TechnOz 
Educational input . If so, how can we turn him to be the machine for 

assessment competency? 

The following is a designed suggestion for Learnability assessment competency, 
which is expected to improve Technology Education planning based on: 

Educational and Psychological Measurement in 
Technology Education Planning 

TOPICAL OUTLINE 

• ATTRIBUTES OF GOOD MEASUREMENT 

1- Validity 

Content validity 
Predictive validity 

2- Reliability 

3- Usability 

• MEASUREMENT IN THE TECHNICAL COGNITIVE DOMAIN 
Published tests in the cognitive domain 

1 - Individual tests of intelligence 

Revised Stanford-Binet intelligence Scale 
Wechsler intelligence Scale for Student turn (WISC) 

2- Group tests of general intellectual ability 

And other aptitude tests 

3- Tests of educational achievement 

4- Tests of creativity 

5- Criterion-referenced achievement tests 
Teacher-developed procedures 

1 - Teacher-made objective achievement tests 

Types of objective test items 
Guides for constructing objective test 

2- Essay tests 

3- Performance tests 

4- Ratings of work samples 

5- observation and rating of typical classroom activity 

• MEASUREMENT IN THE PSYCHOMOTOR DOMIAN 
Published tests of psychomotor skills and physical fitness 
Teacher-developed procedures 
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• MEASUREMENT IN THE AFFECTIVE DOMIAN 
Teacher-developed procedures 

1 -Questionnaires, checklists, and interviews 

2-Observation 



POSSIBLE STUEDENT OBJECTIVE 

Upon completing a course of study, appropriate sections of the student workbook, and 
discussion as necessary, teaching staff member should be able to: 

1. a. Discriminate between measurement and evaluation and indicate the plate of 
measurement in evaluation. 

b. Describe the need for measurement and evaluation in Technical Education. 

c. Explain content validity, predictive validity, reliability and usability as 
applied to educational and psychological measurement. 

2. a. Give an example of each of the five categories of published tests in the 
cognitive domain and indicates a possible effective use and a possible misuse of 
each category of the tests. 

b. State possible uses of each of five categories of teacher-developed 
assessment devices in the cognitive domain. 

c. State some limitations of objectives achievement tests and essay tests in 
terms of content validity, reliability and usability, and describe procedures for 
overcoming the limitation. 

3. a. Explain why much performance testing, rather than using paper-and-pencil 
tests, is done in the psychomotor domain. 

b. Give examples of published tests and teacher-developed procedures in the 
psychomotor domain 

4. a. Explain why there is decreasing use of published assessment devices in the 
affective domain. 

b. Give examples of how students’ attainment of objectives in the affective 
domain is measured. 

c. Examine published tests and accompanying manuals to determine whether 
the particular test has desired content validity and sufficiently high reliability 
for teaching staff member purposes in using it. 



O 

ERLC 



4 5 



Techno-Educational measurement is a tool concerned with ascertaining the quantity, 
extent, or degree of student learning, of teaching effectiveness, or some other facts of 
education. We can measure the student’s level of achievement through administrating 
and scoring an achievement test. Also, we can count the errors in capitalization, 
spelling, punctuation, and sentence structure in a theme. Likewise, the number of 
exercise correct on an arithmetic assignment may be counted. These and other kinds 
of measurements are useful in subsequent evaluation in particular in Technology 
Education assessment e.g. in Colleges of Technology updated curriculum. 

Evaluation ' n education is the process of judging whether the quantity or extent of 
something as measured is acceptable or desirable in terms of some criterion. 
Evaluation thus involves three-steps process of securing information through 
measuring, establishing criteria which to relate the measurement, and making a 
judgment about the relationships. Evaluation stops short of indicating what action 
should be carried out on the basis of the evaluative judgment. For example, a student 
spells 7 of 10 words correctly. Ten of 10 words spelled correctly have been set as the 
criterion for proceeding further in a spelling program. The student hasn’t reached the 
criterion. Now the teacher must decide what to do on the basis of the student’s not 
meeting the criterion. Further information about the instructional program and the 
student may be needed to make wise decision. Although taking action is not a part of 
evaluation, teachers often make the decision regarding instructional matters and take 
actions based on decision. 

Both measurement and evaluation pertaining student learning are required at three 
points in time: 

1- Prior to the start of an instructional sequence so that an appropriate 
instructional program can be arranged for the students (initial evaluation ). 

2- During each sequence of instruction, measurement should facilitate the 
student’s progress (formative evaluation). 

3- At the end of a sequence to determine how well instructional objectives were 
met and to decide on appropriate courses of actions (accumulative evaluation) 

In addition to evaluating student learning, the teacher also needs to evaluate the 
effectiveness of the instructional program for the particular student. Finally, the 
effectiveness of the school’s educational program undoubtedly will improve from 
year to year if the school people probably evaluate it. The tests and other means of 
measurement in the cognitive, psychomotor and affective domains described may be 
used to secure information about student learning and the effectiveness of 
instructional programs. 

ATTRIBUTES OF GOOD MEASUREMENT 

The required three desirable attributes of educational and psychological measuring 
devices are validity, reliability and usability. 

1. VALIDITY 

Validity is a concept that relates the properties of a test to the purpose of testing. A 
test may be valid for one purpose but not for another. Because only the test-user 
knows exactly what the purposes are, the burden of deciding whether or not a test is 
valid in a given situation ultimately rests with the test-user. Three of the main 
purposes of testing are to determine objectives that students are ready to attain, their 
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progress in attaining objectives, and the existent of attainment of objectives at the 
termination of an instructional sequence. Let us relate the purpose of tests to two 
types of validity with which teachers are concerned such as Colleges of Technology 
in this particular case. 

Content validity 

Content validity is concerned with how well the test gives samples from all the 
desired content-knowledge, skills, attitudes that should have been learned. Therefore, 
a content-valid test contains items that draw from each kind of content specified for 
the instructional program. Further, the items should reflect what received emphasis in 
the instructional program, that is, what the student had an opportunity to learn. 

Predictive validity 

Toward the end of a school year, teachers, parents and students are interested in 
deciding which courses the student will take the following year. Student counselors, 
for instance, may use test score as one kind of information to predict how well a 
student will perform if placed in a course, e.g., electronics, computer science, general 
mathematics or production and mechanics. Justification for using a test to select 
individuals for any particular course is provided if there is strong evidence that the 
test has high predictive validity. An example of such evidence is the data collected 
over the years on the same individuals-first their scores on the test itself and then their 
later performance in the course. If the test scores and achievement are highly 
correlated, then one is warranted in making predictions on the basis of scores on the 
test. We note that during any given year, the grades made by students, teacher 
judgments about the student and the student’s expressed interest are useful in 
predicting the student’s achievements the following year. Thus, they have predictive 
validity for this purpose. 

2. Reliability 

Reliability refers to the accuracy of a measurement. Why are accurate, consistent 
measures needed? Why are the teachers and commercial test developers so concerned 
about reliability? An inaccurate measurement does not give a true picture of present 
status and therefore has little or no predictive validity. For example, an unreliable 
reading readiness test administrated early in the first grade is of no value in describing 
students’ present readiness or predicting students’ later achievements in reading. It is 
not helpful in deciding whether a student will profit from beginning reading 
instructions or some time later {such as testing new students reading a technical text.) 
What conditions may result in low reliability of a test? The most important are as 
follows: 

• A test may be unreliable because of poorly constructed items do not 
discriminate between students who possess the knowledge, skill, or other 
attributes being tested and those who do not. In this case a student who knows the 
particular subject matter may get the item wrong, whereas the one who does not 
know it gets the item correct. Stepien, W. J., and Gallagher, S. A. (April 1993): 2- 
28. (ERIC No. EJ 461 126) 

• The length of a test also can be responsible for low reliability. Suppose 300 
words in technical vocabulary have been taught during the year and a test is 
constructed to measure the students writing achievement. One single spelling 
word is a totally inadequate sample of a student’s achievement. In a longer test of 
50 items, we get more dependable scores-there is a better possibility that 
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variations accounted for by chance-guessing, carelessness, inattentiveness and 
other conditions will cancel out. A test can, however, be made so long that 
students become tired or bored and respond unreliably . 

• Subjective scoring of tests and work sample may produce low reliability. 
For example, the teacher may be alert when reading one paper and tired when 
reading another. This may influence the scoring. In this case the scorer’s 
judgements as represented in the rating or scores are not reliable. 

• Inadequate time to complete a test lower reliability in any test where time is 
not a criterion of performance. This is a difficult problem to overcome, for the rate 
at which students respond to a test varies markedly. Nevertheless, if a testing 
session is stopped before some students are finished, the test is not a high 
reliability measure for those who did not complete it. 



How to Overcome Low Reliability 

Returning to the matter of low reliability that results from unreliable scoring, we note 
that carefully prepared criteria and instructions for scoring or rating may prove 
helpful. For instance, the Handbook for Essay Tests, of Educational Progress, 
suggests that students’ essays be rated from 1 to 7, 7 being the highest rating 
(Sequential Tests of Educational Progress, 1957). Norms were based on an earlier 
scoring of essay in which readers scored on the basis of three factors: 

Quality of thought , 50 percent; Style, 30 percent; and Conventions , 20 percent. 

The factors were defined as follows: 

Quality of thought : means “the selection and the adequacy of ideas to the 
supplementary details, and the manner of their organization should reflect their 
coherence derived from the arrangement of parts, (e.g. they should reflect Techno- 
educational input if the test is held in Colleges of Technology) 

Style : means “clearness, effectiveness, and appropriateness, including matters of 
structure diction, emphasis, the means of transition between ideas, and the finer points 
of simplicity, variety, and exactness of expression.” 

Conventions : means “the properties of mechanical form, including grammar and 
usage, capitalization, punctuation, and the mechanical aspects of the structure of 
sentences.” A number of essays are then presented in the Handbook with rating and 
comments to guide the rater in making judgment. Eble, R.L. Criterion-referenced 
measurements^ 1972). 

3. Usability 

In addition to validity and reliability, a number of practical matters dealing with 
usability must be considered particularly when published tests are being widely 
distributed. Usability refers to these factors: (1) the amount of time required to 
administer the test; (2) the amount of preparation or education required to score the 
test; (3) the amount of time required to score the test; (4) the ease of interpreting the 
test results after the scores are obtained; (5) the cost; (6) the mechanical makeup of 
the test. Because of these considerations are quite important, test publishers usually 
provide such information in the test manual, along with the data on the reliability and 
validity of the test. The best source for much of this information is the Mental 
Measurement Yearbook series, prepared by Buros that has been published periodically 
since 1938. Often used in conjunction with Tests in Print and Personality Tests and 
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Reviews, the yearbooks include not only descriptive information about many 
published tests, but also critical reviews of the various tests and a bibliography of 
articles and book about each test. 

Observation 

Observations by teachers who are known to the students and who are performing in 
their usual role should get at typical behaviors in the affective domain. The fact thaj 
the teacher has many, opportunities for observing does noj ensure that assessments 
will automatically be reliable. Systematic procedures for observing and recording 
information are needed. Wilson, B., and Cole, P. 39, no. 4 (1991): 47-64. A few 
points about observing and recording that follow are necessary to teaching staff member ■ 
So what I could say him in this regard is that “To get valid observations, clarify what 
you wish to observe and then identify the behavior that pertains to it. For example, if 
you wish to deal with level of motivation, focus on the student’s behavior dealing 
with initially attending to instructional activities and persisting until the activities are 
completed. This will improve the validity of your observations.” 

Reliability of the observations may be improved by recognising any favourable or 
unfavourable opinions you may have about particular students. Subjectivity, which 
leads to inaccuracies, may be identified by having another teacher observe and rate 
the same students at exactly the same time. In a team-teaching situation, for instance, 
at least two team members may rate each student; thus a more reliable judgement is 
obtained than if a single teacher rated each particular student. 



Reliability may also be improved by counting and recording instances of specified 
behaviours. Jones, J. E. ; (Winter 1994): 23-25; (ERIC No. EJ 498 584) 

The incidence count involves calculating the behaviours of interest as they occur 
throughout the school day. 

• Check any of the following, which are, indicated as potential difficulties 

involved in measurement in the affective domain. 

a. Invasion of privacy. 

b. Low content validity of measurement tools. 

c. Subjective evaluation of observed student behaviour. 

d. Deceitfulness of student. 

e. Incompetence of teachers. 

• Check any of the following which may contribute to effective measurement in 
the affective domain. 

a. Clearly stated objectives. 

b. Holding student self-reports in confidence. 

c. Counting incidences of specified behaviours. 

d. Eliminating the use of questionnaires. 

e. Avoiding use of the individual interview. 
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Validity of assessment competency vs redesigned planning 



Assessment competency can be completed either by formal testing with published and 
teacher-made testing or by observing and scaling students typical classrooms 
behaviour. Pencil-and-paper tests should be supplemented by teacher observation and 
the collection of work samples so that both the optimum and typical achievements 
along with other characteristics of students are sampled. When constructing tests, we 
strive for (1) validity, to assure that the test items sample the content being taught and 
that the test score is useful in prediction; (2) reliability, to secure accurate 
measurements; and (3) usability, to assure that time, and effort are spent wisely. 

Many tests have been standardized in such a way that any individual’s score may be 
compared with the scores made by the standardization sample of students. These tests, 
which thus are norm referenced, are available to test abilities, achievements, and other 
characteristics of student in the cognitive domain. For example, the Wechsler scales 
and the Revised Stanford-Binet Scale are intelligence tests used to test one individual 
at a time. Other intelligence tests commonly used in the schools are administrated 
simultaneously to groups of students. Norm referenced, educational achievement in 
curriculum, areas such as (technical text readability), mathematics, or science. 
Criterion- referenced tests are constructed to measure the attainment of specific 
instructional objectives over short time periods. They may serve as pretests and also 
may be used in diagnosis, which offer a better chance of planning improvement or 
redesigning planning to overcome the deficiency caused by unseen factors. 

Teaching Staff in Colleges of Technology and the designed Tests 

Teacher-made tests, as other measuring devices, should have high reliability and 
content validity. The preparation of good tests requires careful thought in selecting the 
content and constructing the items. Similarly, the securing and assessment of work 
samples and observing typical classroom behaviors of students requires thoughtful 
consideration in order to facilitate student progress, in particular in teaching 
technology. Because teaching technology has got various trends recently, which are 
all controlled by appropriate assessment through well-designed tests or questionnaires 

The Outcome of_ the research pager 

Conclusive Results 
& 

Recommendations 



1. Planning to meet the overall objectives 

There are many different approaches to the definition of technology. There are 
however, no right or wrong definitions, but the workable definitions are the ones that 
are more or less useful. Braveman, H., (1974). The approach that one adopts to define 
anything depends on why one is studying it in the first place. We have to decide 
which approach to technology will be the most useful in helping us understand 
student behavior throughout assessment competency as the first step for planning. So 
simple and systematic job-analysis based on curriculum design has to be done every 
semester to compare the last achievement to the outcome of the newly introduced 
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material. It is one of the most relevant assessment practicalities to be considered to 
improve Technical Education. 

2. Student’s computerized academic behavior and fruitfulness of assessment 

The best place to start computer based Technology Education where assessment is 
expected to be more helpful in educational planning, is the very beginning of 
enrollment where a student can easily state the difference between high school 
learning style and the university level style. It is where a student feels the difference, 
feels that he is in a different place, at a different stage and highly spirited by being 
accepted in a university College of Technology... It is this spirit that has to be 
invested learning wise aiming at a better Techno-educational input. The other reason 
is that the predefined category of studying technology esteems student to actualize a 
better academic performance. 

3. Assessment of Immediate Practical Needs 

Immediate Practical Needs (IPN) have to be assessed in order to harmonize the level 
of student to meet the standard of the assigned material to make him able to assist the 
practicalities of planning and not to fail it. This is not to change the material, but to 
adopt a better approach for student to adapt himself with the course required academic 
technical stuff. Harmony has to begin with resolving conflicts to diminish emerging 
difficulties that hinder setting of Techno-educational priorities and learnability 
afterwards. Proper priorities definitely help staff member understand the hierarchy of 
the material in relation to student capability within the course time span. 

Succeeding to make assessment of immediate practical needs explains the link 
between proper assessment and achieving the easy fruitful follow-up, which achieve 
the following tow wings of appropriate planning at the end: 

• A solid knowledge base for both teaching staff and student. 

• Willingness to use imagination and creative thinking. 

4. Straightening Technology Education curriculum 

It is more than one mistake for a technology teaching staff member not to know how 
to develop or straighten the material given to teach. The type of development and 
straightening have to qualify student to be up to the standard required on the one hand 
and clarify the new advanced issues for the teaching staff member to see them in the 
best scope to master on the other hand. 



5. Required Ongoing Assessment 

If we have a look to the whole content of this paper, we will find out how far 
assessment is needed to have a central focus on Techno-educational planning through 
the full control over student learnability in steps. This would rather highlight the 
skeleton of planning in the wider scope. So what we are interested to end up with is 
the quality of assessment that leads to a refined planning for Technology Education in 
the decision-making bureau for Technology Education, represented for instance in 
General Organisation of Technical Education and Vocational Training. (GOTEVT) in 
Saudi Arabia. 
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6. Direct Access to Technology Education Resources 



In order to have a better results of strategic planning, we have to have a direct access 
to technical educational resources that enables the management and teaching staff to 
carry on applying the curriculum without interruption. Continuity on what is planned 
would rather give a better chance to assessment competency rather than shifting from 
one idea to another to reform a plan. 

7. Technology Advancement and nonstop Planning 

By all means we cannot cope with the technological advancement in the same rhythm 
of inventor. But we do lots of trials in various aspects. We do our best whenever 
possible. So our trials should be based on assessment and planning with reference to 
both student and teaching staff capability to cope with expectations of higher 
education authorities for technology worldwide represented for instance in 
(GOTEVT) in Saudi Arabia, ERIC in US, VET authority in Germany, DTU in 
Denmark and all other similar boards. 

@@@@@ 
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